Pixel interpolation device and pixel interpolation method, and image processing device, and program and recording medium

ABSTRACT

A first group of interpolation calculators performing interpolation by mean preserving interpolation calculation, and a second group of interpolation calculators performing interpolation by interpolation other than the mean preserving interpolation calculation are provided. Each of the interpolation calculators performs interpolation calculation for the missing pixel to calculate an interpolation candidate value (M(h)), and calculates test interpolation values (M(t)) treating pixels in a vicinity of the missing pixel as test pixels. A decider (4) selects one of the plurality of interpolation calculators based on results of the test interpolation. An outputter (5) selects the interpolation candidate value outputted from the interpolation calculator selected by the decider (4), among the plurality of interpolation candidate values (M(h)), and outputs the selected interpolation candidate value. Interpolation can be performed such that the interpolated pixel does not look unnatural, and yet the storage capacity and the data processing amount that are required are relatively small.

CROSS-REFERENCE TO RELATED APPLICATION

The present application is based on PCT filing PCT/JP2018/024444, filedJun. 27, 2018, the entire contents of which are incorporated herein byreference.

TECHNICAL FIELD

The present invention relates to a pixel interpolation device and apixel interpolation method, and an image processing device, and aprogram and a recording medium.

BACKGROUND ART

An image reading device has been realized in which an object to be read(original) is read by a one-dimensional imaging device (line sensor),and image data corresponding to the original is generated. Line sensorsin which a plurality of sensor chips each having a plurality ofphotoelectric conversion elements are aligned in the longitudinaldirection of the line sensor, and are joined together are widely used.

The sensor chips are formed by semiconductor manufacturing technology,so that the intervals between photoelectric conversion elements arerelatively small. In contrast, the line sensor is manufactured byjoining a plurality of sensor chips and fixing them on a board, or in asimilar method. Accordingly, the interval between the photoelectricconversion elements positioned at the mutually adjacent ends of thesensor chips which are joined is relatively large. When such a linesensor is used to read the original to acquire image data, no pixelvalue is obtained at a position between the mutually adjacent ends ofthe sensor chips which are joined. It is necessary to interpolate apixel value for such a pixel.

Also, there are situations in which the pixel value of a pixel signalobtained by reading by means of a photoelectric conversion element isincorrect or unreliable for reasons such as irregularity of thephotoelectric conversion element in the sensor chip, or smear of a partwhich is positioned in the optical path between the photoelectricconversion element in the sensor chip and the original, and which shouldbe transparent. Interpolation is also necessary for such a pixel value.

The pixel at a position where there is no photoelectric conversionelement, and the pixel for which interpolation is necessary because thepixel value is incorrect or unreliable, or for some other reason arereferred to as a pixel with a missing pixel value, or simply as amissing pixel.

Various interpolation methods have been proposed for the interpolationof the missing pixel. Patent reference 1 shows use of pattern matchingfor the interpolation of a missing pixel when the missing pixel is foundto be in an image region in which the pixel value varies periodically.

PRIOR ART REFERENCES Patent References

Patent Reference 1: U.S. Pat. No. 5,699,621 (Paragraphs 0037 to 0068,FIG. 2)

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

In the method disclosed in patent reference 1, it is necessary toprepare in advance an appropriate pattern for each period of the pixelvalue variation. If the number of patterns is insufficient, a pixelvalue which causes the missing pixel to look unnatural may becalculated. Increase in the number of patterns is associated with aproblem of increase in the storage capacity or data processing amount.

Means for Solving the Problem

A pixel interpolation device according to the present inventioncomprises:

a plurality of interpolation calculators each interpolating a pixelvalue of a missing pixel in an input image, based on pixel values ofpixels constituting said input image;

a decider; and

an outputter, wherein

each of said plurality of interpolation calculators interpolates thepixel value of said missing pixel, and outputs the interpolated pixelvalue as an interpolation candidate value, treats a plurality ofnon-missing pixels in a vicinity of said missing pixel as test pixels,interpolates a pixel value of each of said test pixels, and outputs theinterpolated pixel value as a test interpolated value of the test pixel,

the interpolation of said test pixels is performed in a sameinterpolation method as the interpolation of said missing pixel,

said plurality of interpolation calculators include a first group ofinterpolation calculators, and a second group of interpolationcalculators,

said first group of interpolation calculators perform the interpolationby mean preserving interpolation calculation,

said first group of interpolation calculators differ from each other ina number of pixels referenced,

said second group of interpolation calculators perform the interpolationby interpolation calculation different from said mean preservinginterpolation calculation,

said second group of interpolation calculators differ from each other inat least one of an interpolation calculation method and a number ofpixels referenced,

said decider calculates error index values pertaining to respectiveinterpolation calculators, based on absolute difference values betweensaid test interpolated values pertaining to the respective test pixelsin the vicinity of each missing pixel, and the pixel values of the testpixels, and selects one of the plurality of interpolation calculatorsbased on the calculated error index values, and

said outputter selects, for each missing pixel, the interpolationcandidate value outputted from the interpolation calculator selected bysaid decider, among the interpolation candidate values outputted fromthe plurality of interpolation calculators, and outputs the selectedinterpolation candidate value.

Effect of the Invention

According to the present invention, it is possible to performinterpolation by which the interpolated pixel does not look unnatural,and yet the storage capacity and the data processing amount that arerequired are relatively small.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a functional block diagram schematically showing aconfiguration of a pixel interpolation device of a first embodiment ofthe present invention.

FIG. 2 is a schematic diagram illustrating a contact image sensor.

FIG. 3 is a diagram showing details of the joint part between sensorchips forming the contact image sensor shown in FIG. 2 .

FIG. 4 is a diagram showing pixels referenced in the interpolation by amean preserving interpolation calculator shown in FIG. 1 .

FIG. 5 is a functional block diagram showing an example of aconfiguration of the mean preserving interpolation calculator shown inFIG. 1 .

FIG. 6 is a diagram showing pixels referenced in the interpolation by amean preserving interpolation calculator used in a second embodiment.

FIG. 7 is a functional block diagram showing an example of aconfiguration of the mean preserving interpolation calculator used inthe second embodiment.

FIG. 8 is a functional block diagram schematically showing aconfiguration of a pixel interpolation device of a third embodiment ofthe present invention.

FIG. 9 is a diagram showing pixels referenced in the interpolation by amean preserving interpolation calculator shown in FIG. 8 .

FIG. 10 is a functional block diagram showing an example of aconfiguration of the mean preserving interpolation calculator shown inFIG. 8 .

FIG. 11 is a functional block diagram schematically showing aconfiguration of an image processing device of a fourth embodiment ofthe present invention.

MODE FOR CARRYING OUT THE INVENTION First Embodiment

FIG. 1 is a functional block diagram schematically showing aconfiguration of a pixel interpolation device 1 of a first embodiment ofthe present invention.

The pixel interpolation device 1 interpolates pixel values of missingpixels in an image represented by image data Di supplied to an inputterminal 1 a.

The image data Di is obtained, for example, by reading an object to beread (original) by means of an imaging device provided with a contactimage sensor 900 shown in FIG. 2 .

The contact image sensor 900 shown in FIG. 2 is formed by arranging aplurality of sensor chips 902 in a line on a board 904, and joining themtogether. The longitudinal direction of the contact image sensor 900will be the main scanning direction.

FIG. 3 shows, in detail, a joint part between mutually adjacent sensorchips 902.

As illustrated, each of the sensor chips 902 has a plurality ofphoto-electric conversion elements 906.

In each sensor chip 902, the plurality of photo-electric conversionelements 906 are disposed in a line at a constant interval da. On theother hand, the photo-electric conversion elements positioned at theadjacent ends of two sensor chips 902 which are joined are separated bya distance db. Generally, the distance db cannot be made as small as theinterval da.

In that case, it is desirable that the sensor chips are so disposed thatdb is twice da. This is because, if such a relation holds, theinterpolation process can be performed assuming a situation in which onepixel is missing between the sensor chips, making it possible to obtainan accurate result of interpolation. The following description is madeon such a case.

However, db need not be twice da. It is also possible to perform theinterpolation according to the ratio between da and db.

Moreover, the interpolation process described below can be applied, notonly to interpolation of a pixel at a position between sensor chipswhere no photo-electric conversion element is present, but also tointerpolation in a situation in which the pixel value of a pixel signalobtained by reading by means of a photo-electric conversion element inthe sensor chip is incorrect or unreliable. The reason for the pixelvalue being incorrect or unreliable may be irregularity of thephoto-electric conversion element or smear of a part which is in theoptical path and which should be transparent.

“Missing pixels” in the following description cover pixels which are ata position where there is no photo-electric conversion element, andpixels which require interpolation for reasons of the pixel value beingincorrect or unreliable. Pixels which have a pixel value and do notrequire interpolation are referred to as non-missing pixels.

By performing imaging by means of the contact image sensor 900 once, animage signal Ai representing an image of one line is obtained, and byanalog-to-digital conversion of the image signal Ai, image data Di isobtained.

By moving the original in a sub-scanning direction with respect to thecontact image sensor 900, image data Di for a plurality of lines aresuccessively obtained.

In the following description, the image represented by the image data Diis referred to as an input image, and is denoted by the same referencecharacters Di. Furthermore, the pixel value represented by the imagedata Di is referred to as an original pixel value, and is denoted by thesame reference characters Di. Similar rules apply to other images andpixel values.

The pixel interpolation device 1 can be implemented by processingcircuitry. The processing circuitry can be configured of dedicatedhardware, or software, or a combination of hardware and software. Whenit is configured of software, the pixel interpolation device is formedof a microcomputer including a CPU (Central Processing Unit), a DSP(Digital Signal Processor), or the like.

The pixel interpolation device 1 comprises a data storage 2, a pluralityof interpolation calculators 11-1 to 11-N, 21-1 to 21-M, a decider 4, anoutputter 5, and an interpolated data inserter 6.

The data storage 2 stores the image data Di.

The image data Di stored in the data storage 2 is read, and supplied tothe plurality of interpolation calculators 11-1 to 11-N, 21-1 to 21-M,the decider 4, and the interpolated data inserter 6.

Each of the plurality of interpolation calculators 11-1 to 11-N, 21-1 to21-M interpolates the pixel value of each missing pixel in the inputimage Di, and supplies the interpolated pixel value (interpolated value)as an interpolation candidate value P(h) of the missing pixel to theoutputter 5. It also treats a plurality of non-missing pixels in thevicinity (neighborhood) of the missing pixel, as test pixels,interpolates the pixel value of each of the test pixels, and suppliesthe interpolated pixel value (interpolated value) as a test interpolatedvalue P(t) of the test pixel, to the decider 4.

The interpolation of the test pixel is referred to as testinterpolation. The test interpolation is performed hypotheticallyassuming that the pixel value of the test pixel is unknown.

In the interpolation of each missing pixel, pixels in the neighborhoodof the missing pixel are referenced.

In the interpolation of each test pixel (test interpolation), pixels inthe neighborhood of the test pixel are referenced.

The relative positions of the neighboring pixels referenced in the testinterpolation of each test pixel, with respect to the test pixel, areidentical to the relative positions of the neighboring pixels referencedin the interpolation of each missing pixel, with respect to the missingpixel.

The interpolation calculation method used for the test interpolation isidentical to the interpolation calculation method used for theinterpolation of the missing pixel.

The test interpolation in which a plurality of non-missing pixels in thevicinity of each missing pixel are treated as test pixels may bereferred to as test interpolation pertaining to the particular missingpixel.

The plurality of interpolation calculators 11-1 to 11-N, 21-1 to 21-Minclude a first group of interpolation calculators 11-1 to 11-N and asecond group of interpolation calculators 21-1 to 21-M.

The first group of interpolation calculators 11-1 to 11-N consist offirst to N-th mean preserving interpolation calculators 11-1 to 11-N. Nis an integer not smaller than 2.

The first to N-th mean preserving interpolation calculators 11-1 to 11-Nperform interpolation by mean preserving interpolation calculation.

The first to N-th mean preserving interpolation calculators 11-1 to 11-Ndiffer from each other in the number of pixels referenced in theinterpolation.

The operation of the mean preserving interpolation calculators 11-1 to11-N is described below. First, the operation at the time of theinterpolation of a missing pixel is explained.

FIG. 4 shows part of the input image Di, and indicates the positions ofthe pixels referenced in the interpolation by one of the mean preservinginterpolation calculators 11-1 to 11-N, i.e., the n-th mean preservinginterpolation calculator 11-n (n being any of 1 to N). The position ofthe missing pixel MP in the input image Di is assumed to be known inadvance. The non-missing pixels are indicated by “◯”, and the missingpixel MP is indicated by “x”.

Reference characters MS and NA respectively indicate pixel arrays eachconsisting of a plurality of consecutive pixels. The pixel array MSincludes the missing pixel MP, and the pixel array NA does not includethe missing pixel MP. The pixel array MS is referred to as a missingpart, and the pixel array NA is referred to as a non-missing part. Aswill be apparent from the following description, the pixels in themissing part MS and the non-missing part NA are referenced in theinterpolation. The number of pixels constituting the missing part MS andthe number of pixels constituting the non-missing part NA are the same.This number is denoted by k. In the illustrated example, k is 3.

k is preset as a parameter in each mean preserving interpolationcalculator.

Parameters k of mutually different values are set in the plurality ofmean preserving interpolation calculators 11-1 to 11-N, with the resultthat the mean preserving interpolation calculators 11-1 to 11-N differfrom each other in the number of pixels referenced in the interpolation.

The mean preserving interpolation calculator 11-n interpolates the pixelvalue of the missing pixel MP such that the mean value of the pixelvalues of the pixels in the missing part MS is equal to the mean valueof the pixel values of the pixels in the non-missing part NA. That is,the interpolated value P is so determined that the mean value of thepixels in the missing part MS other than the missing pixel MP, and theinterpolated value P of the missing pixel MP equals the mean value ofthe pixel values of the pixels in the non-missing part NA.

FIG. 5 is a functional block diagram showing an example of aconfiguration of the mean preserving interpolation calculator 11-n.

The mean preserving interpolation calculator 11-n comprises a missingpart adder 101, a non-missing part adder 102, and a subtractor 103.

The image data Di read from the data storage 2 is inputted to themissing part adder 101 and the non-missing part adder 102.

The missing part adder 101 determines a sum of the pixel values of thepixels other than the missing pixel MP, among the k pixels included inthe missing part MS, and outputs the sum as a missing part total MQ.

The non-missing part adder 102 determines a sum of the pixel values of kpixels included in the non-missing part NA, and outputs the sum as anon-missing part total SA.

The subtractor 103 subtracts the missing part total MQ from thenon-missing part total SA.

The difference value obtained as a result of the subtraction is used asthe interpolated value P of the missing pixel MP.

The above-described process can be expressed by the followingmathematical equation.P=SA−MQ  (1)

The interpolated value P outputted from the subtractor 103 as a resultof the above-described process is one so determined that the mean valueof the pixel values of the pixels in the missing part MS is equal to themean value of the pixel values of the pixels in the non-missing part NA.

The reason therefor is explained below.

The mean value MSav of the pixel values of the pixels in the missingpart MS is given by:MSav=(MQ+P)/k  (2)

The mean value NAav of the pixel values of the pixels in the non-missingpart NA is given by:NAav=SA/k  (3)

In order for MSav and NAav to be equal to each other, it is sufficientif the interpolated value P is so determined as to satisfy the followingequation.(MQ+P)/k=SA/k  (4)

By transforming the equation (4), the equation (1) is obtained.

Accordingly, if the interpolated value P is determined by the processrepresented by the equation (1), the mean value of the pixel values ofthe pixels in the missing part MS is equal to the mean value of thepixel values of the pixels in the non-missing part NA.

The interpolated value P calculated for each missing pixel in each meanpreserving interpolation calculator 11-n is supplied as an interpolationcandidate value P(h) to the outputter 5.

The mean preserving interpolation calculators 11-1 to 11-N are capableof performing interpolation with a high accuracy in image edge parts andimage parts where the pixel value varies periodically. With respect tothe image parts in which the pixel value varies periodically, the periodof variation (and hence the spatial frequency) of the pixel value forwhich accurate interpolation is possible varies depending on the numberof the referenced pixels. The numbers of pixels referenced in theinterpolation by the mean preserving interpolation calculators 11-1 to11-N are therefore determined according to the range of the period ofthe variation (the range of the spatial frequency) of the pixel valuefor which accurate interpolation is desired.

The mean preserving interpolation calculator 11-n shown in FIG. 5 isalso used for the test interpolation.

The operation performed by the mean preserving interpolation calculator11-n at the time of the test interpolation is identical to theabove-described operation performed at the time of the interpolation ofthe missing pixel MP (calculation of the interpolation candidate valueP(h)). However, at the time of the test interpolation, the interpolationcalculation is performed by treating each test pixel as a missing pixel,and the calculated pixel value P is outputted as the test interpolatedvalue P(t).

The second group of interpolation calculators 21-1 to 21-M consist offirst to M-th interpolation calculators 21-1 to 21-M. M is an integernot smaller than 2.

The first to M-th interpolation calculators 21-1 to 21-M performinterpolation in an interpolation calculation method different from theabove-described mean preserving interpolation calculation.

The first to M-th interpolation calculators 21-1 to 21-M differ fromeach other in at least one of the interpolation calculation method usedfor the interpolation and the number of pixels referenced in theinterpolation.

Examples of the interpolation calculation performed by the first to M-thinterpolation calculators 21-1 to 21-M include linear interpolationcalculation, bi-cubic interpolation calculation, and Lanczosinterpolation calculation.

Like the first to N-th mean preserving interpolation calculators 11-1 to11-N, the first to M-th interpolation calculators 21-1 to 21-M not onlycalculate interpolation candidate values P(h) of each missing pixel, andsupply them to the outputter 5, but also calculate test interpolatedvalues P(t) pertaining to each of the plurality of non-missing pixels inthe vicinity of the missing pixel, and supply them to the decider 4.

The decider 4 receives not only the test interpolated values P(t)supplied from the interpolation calculators 11-1 to 11-N and 21-1 to21-M, but also the pixel values (original pixel values) Di of the testpixels from the data storage 2.

The decider 4 performs evaluation based on the test interpolated valuesP(t) inputted from the interpolation calculators 11-1 to 11-N and 21-1to 21-M, and the original pixel values Di inputted from the data storage2, and calculates error index values indicating the results of theevaluation. The evaluation is performed for each of the interpolationcalculators and for each missing pixel.

For example, the error index value pertaining to each interpolationcalculator is determined by collecting the results of the testinterpolation pertaining to each missing pixel. For example, the errorindex value pertaining to each interpolation calculator is determined bycollecting absolute difference values between the test interpolatedvalues P(t) calculated by the interpolation calculator, for therespective test pixels in the vicinity of the missing pixel, and theoriginal pixel values of the test pixels, over all the test pixels inthe vicinity of the missing pixel.

For example, the above-mentioned error index value may be a valueobtained by dividing an accumulated value of the above-mentionedabsolute difference values over all the test pixels, by the number ofthe test pixels, i.e., an average value (difference average value) ofthe above-mentioned absolute difference values.

If the number of the test pixels for which the test interpolated valuesP(t) are calculated is identical between the interpolation calculators,the above-mentioned error index value may be the accumulated value(difference accumulated value) of the above-mentioned absolutedifference values over all the test pixels.

The above-mentioned error index value represents the suitability of eachinterpolation calculator, i.e., the accuracy of the interpolationcalculation by each interpolation calculator, in the vicinity of eachmissing pixel, and it can be said that the smaller the error index valueis the higher the accuracy (the smaller the degree of the error) is.

For each of the interpolation calculators 11-1 to 11-N and 21-1 to 21-M,the above-mentioned error index value is calculated for each missingpixel.

The decider 4 identifies the interpolation calculator which has yieldedthe smallest error index value, for each missing pixel, and outputs asignal (selection signal) HS indicating the identified interpolationcalculator, to the outputter 5.

As has been described, the interpolated values P calculated by theinterpolation calculators 11-1 to 11-N and 21-1 to 21-M for each missingpixel are inputted as the interpolation candidate values P(h), to theoutputter 5.

The outputter 5 selects the output of the interpolation calculatorindicated by the selection signal HS from the decider 4, among theoutputs (interpolation candidate values) P(h) of the interpolationcalculators 11-1 to 11-N and 21-1 to 21-M, and outputs the selectedoutput as the interpolated value HV of the missing pixel.

The interpolated data inserter 6 receives the image data Di read fromthe data storage 2, and receives the interpolated value HV outputtedfrom the outputter 5, inserts the interpolated value HV pertaining tothe missing pixel outputted from the outputter 5, as the pixel value ofthe missing pixel in the input image Di, and outputs the image data Dufilled with the pixel value of the missing pixel.

The image data Du has the interpolated value HV for the missing pixel,and the original pixel values for the non-missing pixels. That is, thepixel value Du represented by the image data Du is equal to theinterpolated value with regard to each missing pixel, and to theoriginal pixel value Di with regard to each non-missing pixel.

By forming the pixel interpolation device in the manner described above,it is possible to select, as the interpolation calculation for eachmissing pixel, the interpolation calculation whose test interpolatedvalues are the closest to the original pixel values Di, with regard tothe test pixels in the vicinity of the missing pixel. It is thereforepossible to perform the pixel interpolation in which the result ofinterpolation calculation which is most suitable is used according tothe feature of the image (image part in the vicinity of the missingpixel).

As was explained above, the mean preserving interpolation calculators11-1 to 11-N can perform interpolation with a high accuracy in the imageedge parts and in the image parts where the pixel value variesperiodically. With regard to the image parts where the pixel valuevaries periodically, one of the plurality of mean preservinginterpolation calculators which is most suitable is selected accordingto the period of the pixel value variation. In the parts other than theimage edge parts and the image parts in which the pixel value variesperiodically, interpolation by one of the interpolation calculatorsother than the mean preserving interpolation calculators may achieveinterpolation with a higher accuracy. In such a case, the interpolationcalculator which is other than the mean preserving interpolationcalculators and which achieves the interpolation with a higher accuracyis selected. Therefore, in the pixel interpolation device of the presentembodiment, the most suitable interpolation calculator is selectedaccording to the feature of the image.

Also, by including the mean preserving interpolation as theinterpolation calculation, it is possible to reproduce, with a highaccuracy, the image parts periodically varying at a high frequency andedge parts.

Second Embodiment

The overall configuration of the pixel interpolation device 1 accordingto a second embodiment is as shown in FIG. 1 .

The second embodiment differs from the first embodiment in the contentof the process performed in each of the mean preserving interpolationcalculators 11-1 to 11-N.

In the first embodiment, the number of pixels in the missing part andthe number of pixels in the non-missing part which are referenced in theinterpolation by each mean preserving interpolation calculator 11-n areequal to each other. In the second embodiment, the number of pixels inthe non-missing part is larger than the number of pixels in the missingpart.

FIG. 6 shows part of the input image Di, and indicates the positions ofthe pixels referenced in the interpolation by the mean preservinginterpolation calculator 11-n in the present embodiment.

As in FIG. 4 , the position of the missing pixel MP in the input imageDi is assumed to be known in advance. The non-missing pixels areindicated by “◯”, and the missing pixel MP is indicated by “x”.

The missing part MS and the non-missing part NA are, as in FIG. 4 ,pixel arrays each consisting of a plurality of pixels, the missing partMS includes the missing pixel MP, and the non-missing part NA does notinclude the missing pixel MP.

Unlike FIG. 4 , the number of pixels, kb, constituting the non-missingpart NA is larger than the number of pixels, ka, constituting themissing part MS.

In the illustrated example, ka=3, kb=6.

ka and kb are preset as parameters in each mean preserving interpolationcalculator.

The plurality of mean preserving interpolation calculators 11-1 to 11-Ndiffer from each other in the value of at least one of the parameters kaand kb.

The number of pixels referenced in the interpolation is dependent on thevalues of the parameters ka, kb. Therefore, the plurality of meanpreserving interpolation calculators 11-1 to 11-N differ from each otherin the number of pixels referenced in the interpolation.

As in the first embodiment, the mean preserving interpolation calculator11-n interpolates the pixel value P of the missing pixel MP such thatthe mean value of the pixel values of the pixels in the missing part MSis equal to the mean value of the pixel values of the pixels in thenon-missing part NA.

FIG. 7 is a functional block diagram showing an example of aconfiguration of the mean preserving interpolation calculator 11-n usedin the second embodiment.

The mean preserving interpolation calculator 11-n in FIG. 7 is generallyidentical to the mean preserving interpolation calculator 11-n in FIG. 5. A difference is that a multiplier 104 is added.

The multiplier 104 multiplies the output SA of the non-missing partadder 102 by a parameter ratio ka/kb.

The subtractor 103 subtracts the output MQ of the missing-part adder 101from the output (SA·ka/kb) of the multiplier 104.

The difference value obtained as a result of the subtraction is used asthe interpolated value P of the missing pixel MP.

The above-described process can be expressed by the followingmathematical equation.P=SA·ka/kb−MQ  (5)

The interpolated value P outputted from the subtractor 103 as a resultof the above-described process is one so determined that the mean valueof the pixel values of the pixels in the missing part MS is equal to themean value of the pixel values of the pixels in the non-missing part NA.

The interpolated value P calculated for the missing pixel MP is suppliedas an interpolation candidate value P(h) to the outputter 5.

The test interpolation is performed in the same manner as theinterpolation of the missing pixel, and the interpolated values P aresupplied as test interpolated values P(t) to the decider 4.

Incidentally, in FIG. 6 , the non-missing part consists of a pluralityof consecutive pixels. The non-missing part may alternatively consist ofnon-consecutive pixels. For example, the non-missing part may comprise aplurality of mutually consecutive pixels positioned on one side (leftside in the drawing) of the missing pixel MP, and a plurality ofmutually consecutive pixels positioned on the other side (right side inthe drawing) of the missing pixel MP.

By forming the mean preserving interpolation calculators 11-1 to 11-N inthe manner described above, the number of pixels in the non-missing partbecomes large, and the interpolated value of the missing pixel becomesmore natural, compared with the method of the first embodiment.

Third Embodiment

A pixel interpolation device 1 c according to a third embodiment has aconfiguration generally as shown in FIG. 8 .

The pixel interpolation device 1 c shown in FIG. 8 is generallyidentical to the pixel interpolation device 1 in FIG. 1 , but differs onthe following points.

That is, in place of the mean preserving interpolation calculators 11-1to 11-N in FIG. 1 , mean preserving interpolation calculators 13-1 to13-N are provided.

The first to N-th mean preserving interpolation calculators 13-1 to 13-Nall perform the mean preserving interpolation calculation, but differfrom each other in the number of pixels referenced in the interpolation.

Each of the first to N-th mean preserving interpolation calculators 13-1to 13-N, i.e., the n-th mean preserving interpolation calculator 13-n,is similar to the n-th mean preserving interpolation calculator 11-n inthe first embodiment, but differs in the arrangement of the pixelsreferenced in the interpolation, and has a function of determining adifference between a sum of the pixel values of a plurality ofnon-missing pixels positioned on one side of the missing pixel, and asum of the pixel values of a plurality of non-missing pixels positionedon the other side of the missing pixel.

The operation of the mean preserving interpolation calculators 13-1 to13-N is described below, in comparison with the mean preservinginterpolation calculators 11-1 to 11-N in the first and secondembodiments. First, the interpolation of a missing pixel is explained.

FIG. 9 shows part of the input image Di, and indicates the positions ofthe pixels referenced in the interpolation by the mean preservinginterpolation calculator 13-n in the present embodiment.

As in FIG. 4 and FIG. 6 , the position of the missing pixel MP in theinput image Di is assumed to be known in advance. The non-missing pixelsare indicated by “◯”, and the missing pixel MP is indicated by “x”.

As in FIG. 6 , the missing part MS is a pixel array consisting of kapixels, and the missing part MS includes the missing pixel MP.

Unlike FIG. 4 and FIG. 6 , the non-missing part NA comprises a firstpart NL positioned on one side of the missing pixel MP, and a secondpart NR positioned on the other side of the missing pixel MP.

In the following description, the first part NL is referred to as aleft-side part and the second part NR is referred to as a right-sidepart.

Each of the left-side part NL and the right-side part NR is a pixelarray which consists of a plurality of pixels and does not include themissing pixel MP.

The left-side part NL and the right-side part NR consist of the samenumber of pixels, and are formed to be symmetrical with respect to themissing pixel MP.

For example, the number of pixels, kc, constituting each of theleft-side part NL and the right-side part NR is equal to the number ofpixels, ka, constituting the missing part MS.

In the illustrated example, ka=kc=3.

Accordingly, the number of pixels, kb, constituting the non-missing partNA comprising the left-side part NL and the right-side part NR is twicekc.

ka and kc are preset as parameters, in each mean preservinginterpolation calculator.

The first to N-th mean preserving interpolation calculators 13-1 to 13-Ndiffer from each other in the value of at least one of the parameters kaand kc. Accordingly, they differ in the number of pixels referenced inthe interpolation.

Incidentally, in place of ka and kc, ka and kb may be preset asparameters in each mean preserving interpolation calculator.

Like the n-th mean preserving interpolation calculator 11-n in the firstand second embodiments, the n-th mean preserving interpolationcalculator 13-n interpolates the pixel value of the missing pixel MPsuch that the mean value of the pixel values of the pixels in themissing part MS is equal to the mean value of the pixel values of thepixels in the non-missing part NA. However, the non-missing part NAincludes the left-side part NL and the right-side part NR.

FIG. 10 is a functional block diagram showing an example of aconfiguration of the mean preserving interpolation calculator 13-n.

The mean preserving interpolation calculator 13-n in FIG. 10 includes amissing part adder 101, a left-side part adder 105, a right-side partadder 106, a total calculator 107, a multiplier 104, a subtractor 103,and a difference calculator 108.

The missing part adder 101 determines a sum of the pixel values of thepixels other than the missing pixel MP, among the ka pixels included inthe missing part MS, and outputs the sum as a missing part total MQ.

The left-side part adder 105 determines a sum of the pixel values of kcpixels included in the left-side part NL, and outputs the sum as aleft-side partial sum SL.

The right-side part adder 106 determines a sum of the pixel values ofthe kc pixels included in the right-side part NR, and outputs the sum asa right-side partial sum SR.

The total calculator 107 adds the left-side partial sum SL and theright-side partial sum SR together, and outputs the result of theaddition as a non-missing part total SA.

The left-side part adder 105, the right-side part adder 106, and thetotal calculator 107 in combination form a non-missing part adder 102 c.

The multiplier 104 multiplies the output SA of the non-missing partadder 102 c by the parameter ratio ka/kb.

The subtractor 103 subtracts the output MQ of the missing part adder 101from the output (SA·ka/kb) of the multiplier 104.

The difference value obtained as a result of the subtraction is used asthe interpolated value of the missing pixel MP.

The above-described process can be expressed by the followingmathematical equation.P=(SL+SR)·ka/kb−MQ  (6)

As a result of the above-described process, the interpolated value P ofthe missing pixel MP outputted from the subtractor 103 is one sodetermined that the mean value of the pixel values of the pixels in themissing part MS is equal to the mean value of the pixel values of thepixels in the non-missing part NA, based on the partial sums SL, SR andthe missing part total MQ.

The interpolated value P calculated for the missing pixel MP is suppliedas an interpolation candidate value P(h) to the outputter 5.

The difference calculator 108 calculates an absolute difference valuebetween the output (left-side partial sum) SL of the left-side partadder 105, and the output (right-side partial sum) SR of the right-sidepart adder 106. The calculated absolute difference value is sent, as aleft-right difference value DLR, to the decider 4 c.

The left-right difference value DLR calculated for each missing pixel,represents a difference between the sum of the pixel values of thepixels in the left-side part positioned on the left side (the first partpositioned on one side) of the missing pixel, and the sum of the pixelvalues of the pixels in the right-side part positioned on the right side(the second part positioned on the other side) of the missing pixel.

The calculation of the test interpolated values is performed in the sameway as above. That is, like the mean preserving interpolation calculator11-n, each mean preserving interpolation calculator 13-n treats aplurality of non-missing pixels in the vicinity of the missing pixel astest pixels, performs the interpolation calculation for each of the testpixels to calculate the interpolated value P, and supplies thecalculated interpolated value P as a test interpolated value P(t) to thedecider 4 c.

However, the left-right difference value DLR is not calculated for thetest pixels.

The operation of the interpolation calculators 21-1 to 21-M is the sameas in the first embodiment.

The decider 4 c calculates the error index values based on the testinterpolated values P(t) supplied from the mean preserving interpolationcalculators 13-1 to 13-N and the interpolation calculators 21-1 to 21-Min the same way as in the first and second embodiments.

The decider 4 c in the third embodiment differs from the decider 4 inthe first embodiment and the second embodiment in the operation in theevent that the interpolation calculator yielding the smallest errorindex value is one of the mean preserving interpolation calculators 13-1to 13-N.

In the above-mentioned situation, the decider 4 c determines whether theleft-right difference value DLR corresponding to the mean preservinginterpolation calculator having been determined to yield the smallesterror index value is larger than a predetermined threshold value DRLth,and, if the left-right difference value DLR is larger than the thresholdvalue DLRth, the decider 4 c selects a pre-designated interpolationcalculator among the interpolation calculators 21-1 to 21-M, andcommunicates the result of the selection to the outputter 5.

The pre-designated interpolation calculator is, for example, one whichperforms interpolation calculation using pixels proximate, e.g.,adjacent to, the missing pixel. As the interpolation calculation, forexample, linear interpolation calculation or bi-cubic interpolationcalculation is used. When the linear interpolation calculation isperformed, a mean value of a pair of non-missing pixels adjacent to themissing pixel is determined. When the bi-cubic interpolation calculationis performed, a pair of non-missing pixels adjacent to the missingpixel, and pixels adjacent to the above-mentioned pair of non-missingpixels (pixels adjacent to either of the pair of non-missing pixels, andpositioned on the opposite side of the missing pixel) are used asreference pixels.

When there is a large difference in the pixel value between the leftside and the right side of the missing pixel, the mean preservinginterpolation may produce an unnatural pixel value. According to thepresent embodiment, an interpolation calculation method suitable forsuch a situation is used to obtain a more natural pixel value.

In the above-described embodiment, the left-right difference value isdetermined in each of a plurality of mean preserving interpolationcalculators 13-1 to 13-N, and decision is made as to whether theleft-right difference value determined by the mean preservinginterpolation calculator selected based on the error index values, amongthe plurality of mean preserving interpolation calculators 13-1 to 13-N,is larger than the threshold value.

As an alternative, the left-right difference value may be determined inone of the plurality of mean preserving interpolation calculators 13-1to 13-N, and decision may be made as to whether the determinedleft-right difference value is larger than a threshold value.

Also, the left-right difference value may be determined in one of thesecond group of interpolation calculators 21-1 to 21-M, and decision maybe made as to whether the determined left-right difference value islarger than a threshold value.

Furthermore, the left-right difference value may be determined in two ormore interpolation calculators among the plurality of interpolationcalculators, a difference index value may be determined based on the twoor more left-right difference values having been determined, anddecision may be made as to whether the determined difference index valueis larger than a threshold value.

What is essential is that the left-right difference value(s) isdetermined in one or more interpolation calculators among the pluralityof interpolation calculators (13-1 to 13-N, 21-1 to 21-M), and decisionis made as to whether the determined left-right difference value, or adifference index value calculated from the left-right differencevalue(s) is larger than a threshold value.

In the third embodiment, expressions “left” and “right” are used inconnection with the positions of the pixels, and the positions of thenon-missing parts. This, however, is for the sake of convenience, andwhat are meant are one side and the other side of the missing pixel.

That is, it is sufficient if the decider selects a pre-designatedinterpolation calculator among the second group of interpolationcalculators when an absolute difference value between the sum of thepixel values in the first part NL and the sum of the pixel values in thesecond part NR, or a difference index value determined based on theabsolute difference value(s) is larger than a predetermined thresholdvalue.

In the first to third embodiments, the image data outputted from thedata storage 2 may be data representing the luminance of the image, ordata representing color components of the image. In other words, thepixel values processed by the pixel interpolation device may beluminance values or color component values.

When the image is a color image, the pixel interpolation devicedescribed in the first to third embodiments may be provided for eachcolor component value.

In the following, an image processing device is described as a fourthembodiment in which three or more pixel interpolation devices areprovided respectively for three or more color component values, andclipping processes are performed on the color-difference componentvalues.

Fourth Embodiment

FIG. 11 is a functional block diagram schematically showing aconfiguration of an image processing device of the fourth embodiment ofthe present invention.

The image processing device shown in FIG. 11 includes an R pixelinterpolation device 201, a G pixel interpolation device 202, a B pixelinterpolation device 203, a luminance color-difference calculator 205, acolor-difference calculator 206, a clip value determiner 207, a clipprocessor 208, and a color component recoverer 209.

Each of the R pixel interpolation device 201, the G pixel interpolationdevice 202 and the B pixel interpolation device 203 has a configurationsimilar to the pixel interpolation device in the first, second or thirdembodiment. Whereas the pixel interpolation device in the first, secondor third embodiment receives the image data Di and outputs the imagedata Du, the R pixel interpolation device 201, the G pixel interpolationdevice 202, and the B pixel interpolation device 203 in the presentembodiment respectively receive R image data Ri, G image data Gi, and Bimage data Bi, and respectively output R image data Ru, G image data Gu,and B image data Bu.

The R image data Ri, the G image data Gi, and the B image data Bi areobtained, for example, by reading an original by means of an imagingdevice including photo-electric conversion elements having sensitivityto R, G and B light (wavelength components), respectively. Thephoto-electric conversion elements having sensitivity to the R, G and Blight are for example photo-electric conversion elements provided withcolor filters transmitting R, G and B light.

The R pixel interpolation device 201 interpolates pixel values (Rcomponent values) of the missing pixels in the image represented by theinputted R image data Ri. That is, it determines the interpolated valuesof the missing pixels based on the original pixel values (R componentvalues) Ri represented by the R image data Ri, and outputs the imagedata Ru filled with the interpolated values as the R component values ofthe missing pixels.

The R component value Ru represented by the image data Ru is identicalto the interpolated value with regard to each missing pixel andidentical to the original pixel value (inputted R component value) Riwith regard to each non-missing pixel.

The G pixel interpolation device 202 interpolates pixel values (Gcomponent values) of the missing pixels in the image represented by theinputted G image data Gi. That is, it determines the interpolated valuesof the missing pixels based on the original pixel values (G componentvalues) Gi represented by the G image data Gi, and outputs the imagedata Gu filled with the interpolated values as the G component values ofthe missing pixels.

The G component value Gu represented by the image data Gu is identicalto the interpolated value with regard to each missing pixel, andidentical to the original pixel value (inputted G component value) Giwith regard to each non-missing pixel.

The B pixel interpolation device 203 interpolates pixel values (Bcomponent values) of the missing pixels in the image represented by theinputted B image data Bi. That is, it determines the interpolated valuesof the missing pixels based on the original pixel values (B componentvalues) Bi represented by the B image data Bi, and outputs the imagedata Bu filled with the interpolated values as the B component values ofthe missing pixels.

The B component value Bu represented by the image data Bu is identicalto the interpolated value with regard to each missing pixel, andidentical to the original pixel value (inputted B component value) Biwith regard to each non-missing pixel.

The pixel interpolation performed in each of the R pixel interpolationdevice 201, the G pixel interpolation device 202 and the B pixelinterpolation device 203 may be identical to the pixel interpolation inthe pixel interpolation device of any of the first, second and thirdembodiments. However, the number of the interpolation calculatorsincluded in the pixel interpolation device, the interpolationcalculation method used in the interpolation in each interpolationcalculator, and the number of pixels referenced in the interpolation maybe different between the color components. This makes it possible toperform interpolation, for each color component, more suitable to thefeature of the image of each color component.

The luminance color-difference calculator 205 converts the R componentvalue Ru, the G component value Gu, and the B component value Bupertaining to the same pixel in the image data Ru, Gu and Bu, into aluminance value Yu and color-difference component values Cbu, Cru, andoutputs the luminance value Yu and the color-difference component valuesCbu, Cru having been obtained by the conversion.

For the conversion, known conversion formulae are used. For example,when the color component values are represented by 8 bits, the followingformulae are used.Yu=0.257·Ru+0.504·Gu+0.098·Bu+16Cbu=−0.148·Ru-0.291·Gu+0.439·Bu+128Cru=0.439·Ru−0.368·Gu−0.071·Bu+128

The color-difference calculator 206 converts the R component value Ri,the G component value Gi and the B component value Bi pertaining to thesame pixel in the image data Ri, Gi and Bi, into color-differencecomponent values Cbi, Cri, and outputs the color-difference componentvalues Cbi, Cri having been obtained by the conversion. For theconversion to the color-difference component values Cbi, Cri, knownformulae for conversion from RGB to YCbCr may be used. However, since Y(luminance value) is not used, it need not be calculated. When the colorcomponent values are represented by 8 bits, the following formulae areused.Cbi=−0.148·Ri−0.291·Gi+0.439·Bi+128Cri=0.439·Ri−0.368·Gi−0.071·Bi+128

The clip value determiner 207 designates pixels within a predeterminedarea of the input image, as pixels to be referenced for the calculationof clip values (clip value calculation reference pixels), and determinesthe clip values from the Cb component value and the Cr component valueobtained by the conversion at the color-difference calculator 206 forthe clip value calculation reference pixels.

For example, the clip value determiner 207 determines the clip valuesfor each missing pixel. In that case, the predetermined area mentionedabove is an area consisting of pixels in the neighborhood of the missingpixel.

The neighborhood of the missing pixel means, for example, an area inwhich the distance from the missing pixel is not more than apredetermined value.

As the clip values, an upper clip value and a lower clip value aredetermined for each of the Cb component and the Cr component.

For example, the maximum value among the color-difference componentvalues Cbi of the clip value calculation reference pixels, or a valueclose to the maximum value is determined as a Cb component upper clipvalue Cbmax, the minimum value among the color-difference componentvalues Cbi of the clip value calculation reference pixels, or a valueclose to the minimum value is determined as a Cb component lower clipvalue Cbmin, the maximum value among the color-difference componentvalues Cri of the clip value calculation reference pixels, or a valueclose to the maximum value is determined as a Cr component upper clipvalue Crmax, and the minimum value among the color-difference componentvalues Cri of the clip value calculation reference pixels, or a valueclose to the minimum value is determined as a Cr component lower clipvalue Crmin.

A “value close to the maximum value” is, for example, a value at apredetermined place when the color-difference component values arearranged in the order of magnitude. The predetermined place may bedetermined according to the number of clip value calculation referencepixels. For example, it may be determined at an ordinal number matchingthe number corresponding to a predetermined proportion in the clip valuecalculation reference pixels.

Similarly, a “value close to the minimum value” is, for example, a valueat a predetermined place when the color-difference component values arearranged in the order of smallness. The predetermined place may bedetermined according to the number of clip value calculation referencepixels.

The clip processor 208 performs clipping process on the color-differencecomponent values Cbu, Cru, using the upper clip value and the lower clipvalue determined by the clip value determiner 207, respectively as anupper limit and a lower limit, and outputs clipped color-differencecomponent values Cbe, Cre.

The color component recoverer 209 performs conversion from thecolor-difference component values Cbe, Cre outputted from the clipprocessor 208 and the luminance value Yu outputted from the luminancecolor-difference calculator 205, to the color component values, andoutputs the R, G, B component values Ro, Go, Bo obtained by theconversion.

For the conversion, known formulae may be used. For example, when Yu,Cbe, Cre are represented by 8 bits, the following formulae may be used.Ro=1.164·(Yu−16)+1.596·(Cre−128)Go=1.164·(Yu−16)−0.391·(Cbe−128)−0.813·(Cre−128)Bo=1.164·(Yu−16)+2.018·(Cbe−128)

In the above example, the clipping process is performed on the Cbcomponent value and the Cr component value which are generated byconversion from RGB to CbCr. However, the clipping process may beperformed on a Pb component value and a Pr component value which aregenerated by conversion to YPbPr. Also, the clipping process may beperformed on a U component value and a V component value generated byconversion to YUV.

What is essential is that the clipping process is performed oncolor-difference components generated by conversion into a luminancecomponent and color-difference components.

With such an arrangement, the luminance is not altered, while generationof false colors can be restrained.

In the above example, the clip values include the upper clip value andthe lower clip value. Just one of the upper clip value and the lowerclip value may be used.

By forming the pixel interpolation device in the manner described above,interpolation calculation is performed for each of the R, G, and B colorcomponents independently, and it is possible to restrain generation offalse colors in the missing pixel part in the color image. Also, it ispossible to prevent the interpolated pixels in the color image frombecoming unnatural.

The invention has been described as a pixel interpolation device and animage processing device. A pixel interpolation method performed in theabove-described pixel interpolation device, and an image processingmethod performed in the above-described image processing device alsoform part of the present invention.

Also, a program for causing a computer to execute the processes in theabove-described pixel interpolation device or the pixel interpolationmethod, and a program for causing a computer to execute the processes inthe above-described image processing device or the image processingmethod are part of the present invention.

Furthermore, a computer-readable recording medium in which theabove-mentioned program(s) is recorded is also part of the presentinvention.

REFERENCE CHARACTERS

1, 1 c: pixel interpolation device; 2: data storage; 4, 4 c: decider; 5:outputter; 6: interpolated data inserter; 11-1 to 11-N, 13-1 to 13-N:mean preserving interpolation calculator; 21-1 to 21-M: interpolationcalculator; 101: missing part adder; 102: non-missing part adder; 103:subtractor; 104: multiplier; 105: left-side part adder; 106: right-sidepart adder; 107: total calculator; 108: difference calculator; 201: Rpixel interpolation device; 202: G pixel interpolation device; 203: Bpixel interpolation device; 205: luminance color-difference calculator;206: color-difference calculator; 207: clip value determiner; 208: clipprocessor; 209: color component recoverer.

The invention claimed is:
 1. A pixel interpolation device comprisingprocessing circuitry configured as: a plurality of interpolationcalculators each interpolating a pixel value of a missing pixel in aninput image, based on pixel values of pixels constituting said inputimage; a decider; and an outputter, wherein each of said plurality ofinterpolation calculators interpolates the pixel value of said missingpixel, and outputs the interpolated pixel value as an interpolationcandidate value, treats a plurality of non-missing pixels in a vicinityof said missing pixel as test pixels, interpolates a pixel value of eachof said test pixels, and outputs the interpolated pixel value as a testinterpolated value of the test pixel, the interpolation of said testpixels is performed in a same interpolation method as the interpolationof said missing pixel, said plurality of interpolation calculatorsinclude a first group of interpolation calculators, and a second groupof interpolation calculators, said first group of interpolationcalculators perform the interpolation by mean preserving interpolationcalculation, said first group of interpolation calculators differ fromeach other in a number of pixels referenced, said second group ofinterpolation calculators perform the interpolation by interpolationcalculation different from said mean preserving interpolationcalculation, said second group of interpolation calculators differ fromeach other in at least one of an interpolation calculation method and anumber of pixels referenced, said decider calculates error index valuespertaining to respective interpolation calculators, based on absolutedifference values between said test interpolated values pertaining tothe respective test pixels in the vicinity of each missing pixel, andthe pixel values of the test pixels, and selects one of the plurality ofinterpolation calculators based on the calculated error index values,and said outputter selects, for each missing pixel, the interpolationcandidate value outputted from the interpolation calculator selected bysaid decider, among the interpolation candidate values outputted fromthe plurality of interpolation calculators, and outputs the selectedinterpolation candidate value.
 2. The pixel interpolation device as setforth in claim 1, wherein said mean preserving interpolation calculationinterpolates the pixel value of the missing pixel such that a mean valueof pixel values of pixels in a missing part consisting of a plurality ofpixels which include the missing pixel is equal to a mean value of pixelvalues of pixels in a non-missing part consisting of a plurality ofpixels which do not include the missing pixel.
 3. The pixelinterpolation device as set forth in claim 2, wherein a number of pixelsin said missing part and a number of pixels in said non-missing part areequal to each other.
 4. The pixel interpolation device as set forth inclaim 3, wherein, in the mean preserving interpolation calculation, asum of pixel values of the pixels other than the missing pixel, in saidmissing part is determined as a missing part total, a sum of pixelvalues of the pixels in said non-missing part is determined as anon-missing part total, and a difference value obtained by subtractingsaid missing part total from said non-missing part total is determinedas the pixel value of said missing pixel.
 5. The pixel interpolationdevice as set forth in claim 2, wherein a number of pixels in saidnon-missing part is larger than a number of pixels in said missing part.6. The pixel interpolation device as set forth in claim 5, wherein, inthe mean preserving interpolation calculation, a sum of pixel values ofthe pixels other than the missing pixel, in said missing part isdetermined as a missing part total, a sum of pixel values of the pixelsin said non-missing part is determined as a non-missing part total, anda difference value obtained by subtracting said missing part total froma product of a ratio of the number of pixels in said missing part to thenumber of pixels in said non-missing part and said non-missing parttotal is calculated as the pixel value of said missing pixel.
 7. Thepixel interpolation device as set forth in claim 2, wherein saidnon-missing part referenced in the interpolation by at least one meanpreserving interpolation calculator among said plurality of meanpreserving interpolation calculators comprises a first part positionedon one side of said missing pixel and a second part positioned on theother side of said missing pixel, and when an absolute difference valuebetween a sum of pixel values of the pixels in said first part and a sumof pixel values of the pixels in said second part, or a difference indexvalue determined based on said absolute difference value is larger thana predetermined threshold value, said decider selects a pre-designatedinterpolation calculator among said second group of interpolationcalculators.
 8. The pixel interpolation device as set forth in claim 7,wherein said pre-designated interpolation calculator is an interpolationcalculator performing linear interpolation calculation or bi-cubicinterpolation calculation.
 9. The pixel interpolation device as setforth in claim 8, wherein said linear interpolation calculation isperformed referencing non-missing pixels adjacent to the missing pixel.10. The pixel interpolation device as set forth in claim 8, wherein saidbi-cubic interpolation calculation is performed referencing firstnon-missing pixels adjacent to said missing pixel, and secondnon-missing pixels adjacent to said first non-missing pixels andpositioned on the side opposite to said missing pixel.
 11. An imageprocessing device wherein said input image is a color image representedby three or more color component values, said image processing devicecomprising: three or more pixel interpolation devices respectivelyprovided for the color component values, each performing interpolationof the corresponding color component value, a luminance color-differencecalculator to convert color component values outputted from said threeor more pixel interpolation devices into a luminance value and first andsecond color-difference component values; a color-difference calculatorto convert the color component values of said input image into first andsecond color-difference component values; a clip value determiner todetermine at least one clip value from the first and secondcolor-difference component values obtained by the conversion at saidcolor-difference calculator; a clip processor to limit the first andsecond color-difference component values obtained by the conversion atsaid luminance color-difference calculator, using said at least one clipvalue determined by said clip value determiner; and a color componentvalue recoverer to convert the first and second color-differencecomponent values limited by said clip processor, and the luminance valueobtained by the conversion at said luminance color-differencecalculator, into color component values, wherein each of said pixelinterpolation devices is the pixel interpolation device as set forth inclaim 1, and performs interpolation using the corresponding colorcomponent values as the pixel values.
 12. The image processing device asset forth in claim 11, wherein said at least one clip value determinedby said clip value determiner includes an upper clip value and a lowerclip value.
 13. The image processing device as set forth in claim 11,wherein said clip value determiner determines said at least one clipvalue from the first and second color-difference component valuesobtained by the conversion at said color-difference calculator for thepixels in a predetermined area in said input image.
 14. The imageprocessing device as set forth in claim 13, wherein said predeterminedarea is an area consisting of pixels in a neighborhood of said missingpixel.
 15. The image processing device as set forth in claim 11, whereinsaid first color-difference component value obtained by the conversionat said luminance color-difference calculator, as well as said firstcolor-difference component value obtained by the conversion at saidcolor-difference calculator is a Cb component value, and said secondcolor-difference component value obtained by the conversion at saidluminance color-difference calculator, as well as said secondcolor-difference component value obtained by the conversion at saidcolor-difference calculator is a Cr component value.
 16. A pixelinterpolation method comprising: performing a plurality of interpolationcalculation processes each interpolating a pixel value of a missingpixel in an input image, based on pixel values of pixels constitutingsaid input image; wherein each of said plurality of interpolationcalculation processes interpolates the pixel value of said missingpixel, and outputs the interpolated pixel value as an interpolationcandidate value, treats a plurality of non-missing pixels in a vicinityof said missing pixel as test pixels, interpolates a pixel value of eachof said test pixels, and outputs the interpolated pixel value as a testinterpolated value of the test pixel, the interpolation of said testpixels is performed in a same interpolation method as the interpolationof said missing pixel, said plurality of interpolation calculationprocesses include a first group of interpolation calculation processes,and a second group of interpolation calculation processes, said firstgroup of interpolation calculation processes perform the interpolationby mean preserving interpolation calculation, said first group ofinterpolation calculation processes differ from each other in a numberof pixels referenced, said second group of interpolation calculationprocesses perform the interpolation by interpolation calculationdifferent from said mean preserving interpolation calculation, saidsecond group of interpolation calculation processes differ from eachother in at least one of an interpolation calculation method and anumber of pixels referenced, said pixel interpolation method furthercomprises: calculating error index values pertaining to respectiveinterpolation calculation processes, based on absolute difference valuesbetween said test interpolated values pertaining to the respective testpixels in the vicinity of each missing pixel, and the pixel values ofthe test pixels, and selecting one of the plurality of interpolationcalculation processes based on the calculated error index values, andselecting, for each missing pixel, the interpolation candidate valueoutputted from the selected interpolation calculation process, among theinterpolation candidate values outputted from the plurality ofinterpolation calculation processes, and outputting the selectedinterpolation candidate value.
 17. A non-transitory computer-readablerecording medium storing a program for causing a computer to executeprocesses in the pixel interpolation method as set forth in claim 16.